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ABSTRACT 

Various aspects of the goals and processes of the 
evaluation of remedial programs are discussed in this paper. The 
first section discussses assumptions related to remedial program 
evaluation in California's conmiunityco and highlights two 

perspectives^ iDased on whether the remedial' programs satisfy internal 
and/or external criteria. Next, a review is provided of currently 
available information on remedial programs and the elements related 
to effectiveness in successful programs. The following section offers 
assumptions about factors contributing to the program effectiveness 
at the state or system level, the institutional levelv the 
programmatic level, and with respect to students and learning in 
remedial programs. Next> activities are suggested contributing to 
program evaluation arid a model is presented reconimending activities 
arid areas of _ irivest igat ion . I ssues , problems^ and priority concerns 
aretheri butliried. wi th respect to the role of management in program 
evaluatiori. Firially^ a check list of factors influencing efficiency 
arid effectiveness is provided, covering comprehensiveness, 
iristitutional factors, and student performance. (HB) \ 
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ASSUMPTIONS RELATED TO THE EVALUATION 
OF REMEDIAL PROGRAMS IN THE 
CALIFORNIA COiTOITY COLLEGES 

The current climate of the Caiifornic eoiranuhity Colieges 
is characterized by uncertainty; uncertainty, about which 
students to serve, how to serve them, and how effective are 
the programs that serve them. This uncertainty is reflected 
in mandates for accountability in services, courses and programs. 
A key emerg-ing priority is the evaluation of the remedial 
programs in higher education. 

Effective remedial education has represented a commitment 
to 'educational opportunity that bridges the gap between under- 
X^repared students and successful college level work. 

in the past decade ail segments of higher education have 
developed remedial programs to assist the larger number of 
anderprc^pared students to succeed in college courses. Since 
the frajority of the students enrolled in colleges are deficient 
in skills of reading writing and math remedial programs have 
become one of the fastest growing areas of curriculum^ 

The priority of the high risk student and how to increase 
his success in college is how being joined by a growing concern 
for remedial courses and programs and the question is emerging 
of how high a risk are these courses? Many issues surround 
this trend in higher education instruction. The issues have 
to be replaced with specific information. 

The central question is what is the evaluation of remedial 
programs to accomplish? Since program evaluation serves 
different purposes, a rationale for lihe program evaluation needs 
to be made. Two key issues surrounding remedial programs in 
pdstsecdhdary education suggest one perspective - that of modifyinq 
to fit external criteria. What do programs cost? And what is 
the basic rationale for the investment? 
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A second perspective - modifying to fit internal 
criteria - suggests a different set of questions, 

i . What are the key elements of successful remedial programs? 
li How can effective programs be identified? 

3. How can ineffective programs be replaced or terminated? 

4. What kint: of evidence is available to identify effective 
or ineffective programs? 

5. What kind evidence are we to provide of the significant 
benefits of our courses and programs at a reasonable cost? 

6. What reliable evaluation practices must be established at 
the institutions? 

A program evaluation model based on these questions will 
serve three purposes: (1) it will provide information to 
decision makers, (2) it will provide a basis for improving 
existing programs, and (3) it will provide for additional 
success of the participants in the programs. will allow 

for a balance between external and internal criteria, 

INFORMATION NOW AVAILABLE : 

Model remedial programs abound in the community college 
system and tell us the what and how abodt efficiency and 
effectiveness. For example, Friedlander in Should Reme diation 
be Mandatory , identifies three elements of effectiveness in 
successful remedial programs: (1) they provide the under- 
prepared student with a better opportunity to improve academic 
achievement: (2) they enable maintenance of high academic 
standards by better preparing deficient students to do 
subsequent course work, and (3) they increase revenue due to 
higher retention and course enrollment. 
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Existing practices and information need to be incorporated 
into new models; 

The Learning, Assessment^ Reteritibri Cbhsbrtium bf 
California, (LARC) has collected information on Learning Skills 
Programs in the Galifornia Cbmmuhity Colleges, The LARC Program 
Guides (1981-82 and 82-83) describe the prominent activities and 
trends in bver twenty five cbllege programs. 

The Chancellbr's Office, California Communtiy Colleges, and 
the Western Association Accrediting Commission for Cbmmunity and 
Juriibr Cblleges have produced a series of papers resulting from 
a three year project to improve evaluation and planning in 
Community Cblleges. A Project Working Paper on Measuring Community 
College learner Outcomes defines the language of outcbme evaluation 
and reviews ways in which the outcome evaluation approach is 
currently in use. 

The LARC Consortium, tbo ^ has develbped a mbdel that depicts 
the methods by which outcome data can be applied to administrative 
decisibns accbrding tb Tillery (Measuring Community College 
Outcomes) . Currently, the Consortium is developing a schematic 
to collect information for following up assessed students . This 
schematic will be disseminated in May, 1984. 

Definitions of key terms relating to remediation were 
developed in 1982 in a statewide charrette process co-sponsored 
by the LARC Cbhsbrtium and the Califbrhia Association of Community 
Colleges. Definitions of remediation, matriculation, and AA degree 
course work have beeJi prepared by a Task Force on Academic Quality 
(Office of Chancellor, California Community Colleges), Statements 
on proficiencies in English and mathematics have been developed 
by the Academic Senates of the California Community Cblleges / the 
California State University and the University of California. • 
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ASSUI^PT I ONS A BO UT RENED I AL P TO ms 

What have the remedial programs in the Caiifbrnia 
Comitiunity Colleges emphasized and what directions need to be 
considered? Although a thorough analysis of these questions 
probably will be undertaken, current research and practice 
suggest some assumptions about the efficiency and effectiveness 
of remedial programs in the community colleges. These assumptions 
may provide a basis for examining remedial programs . 

The information about efficiency and effectiveness of programs 
is presented in four categories of assumptions: (i) assumptions 
about remedial programs in a broad system or state framework; 

(2) assumptions about remedial programs at an institutional level; 

(3) assumptions about remedial programs themselves, and (4) assump- 
tions about students and learning in remedial programs. 

I . Sys tem or state: Assumptio as^j^iout^-Programs : 

.Since 1973 there has been a steady increase in proportion 
of colleges with successful developmental education 
programs (Friediander ) . 

.The successes of community college remedial programs aire not 
a iTiatter of diversity or local needs. • 

^Because of the breadth and depth of community college programs 
in skills development, community collegfes can provide more 
efficient and effective learning than business or industry. 

a ; Institution: Assumptions About Programs : 

.The college needs to be perceived as a total delivery i^ystem 
of remediation. 

.Academic standards are reinforced by appropriate assessment/ 
placement activities. 

.The balance between open access and academic quality can be 
based oh three elements: . • 

-student competencies 
-learner outcome 
-academic standards 

. Competencies will be a key institutional term. Similar 
vocabulary terjrts are perf ormanee and p rofic iency . - 
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III;, Prograro^^^wpfciohs : 

;The two key elements of program evaluation are efficiency 
and effectiveness. 

.The single remedial course is the least effective of all of 
the remedial ef forts ; . 

.The delivery of educar.ibh is becoming more sharply prescriptive 
ahd_ individualized the term individualized program is less 
misleading than individualized instruction . 

IV. Students and Remedial Pro<?r^ms : 

.Success iTi college is dependent on cognitive and effective 
skills- 

. Successful students in remedial programs do not receive a 
lower quality of educatibh- 

. Applying basic skills to a problem of personal context increases 
learning probability; Since we do not know of a preferred 
hierarchy of basic skills to be learned, ^n appropriate focus 
may be on ability to improve pa^'formance in given situations. 

ACTIVITIES CONTRIBUTING TO PROGR AM EVALUAT I O N : 

Certain focuses seem more relevant than others. For example, 
because of the diversity of programs and college clientele^ 
a comparison of one program to another is not useful; Based 
on these assumptions, the following activities might be useful 
and unique to the community; colleges : 

.A descriptidh of. effective p>rdgrams with an analysis of the 
breadth and depth of prdgraris that can serve emerging needs 
in industry. 

.A description of promising practices in remedial programs. 

.Any information on student competencies in remedial programs. 

.Ah analysis of how the coiriinunity colleges can provide total 
delivery systems in remediation. 

.Alternatives to the single remedial course. 

.Cost effective practices in individualized instruction'; 

.Applied remediation to personal and professional context. 
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In the. absence of a formal set of , guidelines , the California 
Community Colleges need to begin program evaluation activities. 
What can we reasonably expect ' a process of program evaluation to 
include? What factors or information need to be ihcludea to make' 
up a strong statement of purpose arid outcome iri the evaluation of 
remedial programs? Drawn from current information, the following 
questions suggest a number of pdteritial activities ^nd areas of 
investigation . 



QUESTIONS/ISSUES 

FOR THE EVALUATION OF REMEDIAL PROGRAMS IN THE COMMUNITY COtlEGES 







pHitosoPHy 

Goal Setting 


WHO 


WHAT 


HOW 


Course 


WhQ_are_the rernedlal 
students? Hpw_ many? 
At._what levels of 
remediation? 


Does remediatibri 
encompass only basic 

svnis? 


How does renediatioc 
fit. in with total 
college? 


Program 


Who linRs the 
remedial program 
to total college 
program? 

C?-::_aD_ Inst 1 tut ipnal 
emphasis shift to 
competencies? 
Should it? 


What.is.tbe linkage, 
of remedial .programs 
to college mission? 

What is the emphasis 
to be* On short 
duration butcbmes 
or long duration 
butcbmes? 


How high a risk are y 
the remedial courses / 
and prbgrans? y/ 

How do We sake a hisp, 
risk, course -a low risk 
coarse? 

With fixed funding 
how do we remediate 
al 1 who need it? 


Student 


Do saccessful 

students in 

remedial pi^P^t^^ 
receive a lower 
qual Ity of 
education? 


What are the newer 
developments in the 
assessment of 
learner butcbmes? 


How much contnit*ent___ 
is there. to deselQPi^ 
an uoderstaodi.PQ of 
adult learning? 

How broad a range of 
effects on the student 
will be ■easured? 


OPERATIONAL 
Program _ 
Development 


WHO 


WHAT 


HOW 


Course 


Who does the 
course serve? 
(Wha* are their 
heeds?) 


What coherent method- 
ologies frame 

remedial instruction. 


How much variation 
exists bctweer; sa« 
course? 
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UPERATIONAL 








Prbjrlin 
Develdpneitt 








Cohtinued 


WHO 


WHAT 

— - 


Hoy 

- 


Program 


Hho sets the goals? 


What is a program? 


How does the program 




Wliat IS the oreadtn 


fulfill degree 






and (lepiii ot piouraini 


recjuireinents? 






(]}iat indiviaiidlized 


How iiiiich variation 






instruction is 


exists between. 






avaiioDict 
















BQflt_drc..lllc_ . 

adyantageiqf.a.total 


Hpw.nychjariation _ 






UniTlSQ rclncQlal 


Avlcfc hfifuppn ^niir^p 






approach versus a 


in same iirbgram? 






aingic CQUrSc; 












Student 


Do students iinder- 


NHat liappeiis to 


How is stiideht ... 




starid tte piirpose 


Students after ttiey 


success iiieasured? 




and goals of the 


leave the course? 






prbgrain 








Do stodents.atece 








iD.a.sequeDce of... 








learning into more 








advanced courses? ' 






liiirinfMPNtAi 

UUlAiLnCni/^t 








Program 








Review 


OHO 




m 


Course 


Wiio are the success- 


Whatjre.measures of 


How many students drop 


ful students?. 


effectiveness? 


out? ^ 






What noms [internal 


How mariy conplete 






and external) will be 


successfully? 






ised to measure 








performance to? 


How does the course 






iinprjve acadeiiiic 




r 




achievement? 
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JlJDGENENTAL 
Program 

Continued 


mo 


WHAT 


HOW ' 


^ Program 


Do the benefits 
of reniediatidn: 
tp.the.studeot.... 
justify the costs? 


Sbat-are sdliie - 
promising practices - 
in remedial programs? 

Uhat does it cost? 

l^s^CHi lege wruing 
positively affect 
the development of 
writing abilities? 


iiow iny students 
persist from course 
to course? 


, Student 
■ 


DojeiD^dial programs 
merely keep students 
in college pr_con-_ 
tribute. to academic 
success? 


What. are students - 
attitudes Jbout -What 
is being tcught? 


Kow-do studnets 
berfbrin iii other 
progris after BKy 
leave? 

(tow canjrt describe, 
the impact.of rendl- 
ation on the stvdentf 


CIWPETENCY 
Effectivenesi 


WHO 


WHAT 


m 


Course 




Wh.at.is_ readability 
level of text? 




Program 




What definition of 
outcome will be used: 
eig.i large scale 
fihdihg^i.iiiipact, 
prodflctivtty? 




Student 1 

i 


/ 

U what level dp., 
students read and 
irite? 


Hhatjests can..be.used 
to provide jn.fp.rmation 
to measure student 
progress along an 
ability continuun? 
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MANAGEMENT: CONSIDERATIONS FOR 
THE EVALUATIpN OF REMEDIAL PROGRAMS 

The central cjuestioris to be considered in the evaluation 
of remedial programs are h ow and what . The primary issue is 
that remediatibh increasingly is entangled in questions of 
institutional mission aipd purpose rather than learning and 
teaching. As a result^ state level offices have become impatient 
and intrusive in this process. The basic p.ssumptioh that has 
not been underscored effectively in that we have been conducting 
evaluation of remedial programs with documented results in 
adjustments of remedial programs. Courses have been dropped or 
adjusted; new courses have been added; delivery systems adjusted^ 
support services added. Some observers, including Donald K. Smith 
(Marcus, 1983) . believey that the greatest safeguard against an 
increased state role is for the institutions themselves to 
strengthen their own evaluation activities. 

WHAT IS THE EVALUATION TO ACCQMSLISH ? 

At a recent conference for postsecondary education in 
California the consensus of the representatives from all segments 
was that the evaluation 6f remedial programs will be to improve 
programs; tp. adjust^ hot to reduce or diminish. Current informa- 
tion and research suggests three reasons why colleges cannot 
reduce remedial prdgraitis : 

(1) The litearacy problems of students involve a 
majority of«all colleges. 

(2) The typical .profile bf.the community college 
students reflects a wide range of missinig 
acadeiric skills. 

(3) The growing demands for the acquisition of 
higher, skills^^ompetencies . 

/ 
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A recent riatibhal study by Dr, John Rbueche> University, of 
Texas, provides specific information: 

The literacy skills of graduating high school students 

is continuing to decline ; No state 99iT^Pf tencies exam 

in the nation requires a graduating demonstrated competency 

above 9th grade reading; 

All post secondary institutional types from the graduate 
research institution to the cdmiriunity college provide 
remedial assistance in.sbme form to students in the same 
propdrtidri around 50%. 

The community cbiiege student typically earned C grades 
or lower in high school. Few have had geometry^ secofid 
year algebra, English composition, a foreign language^ 
or a year of science. 

The gap in needed literacy skills is increasing^ By the 
l990*s anyone who doesn't have at least av 12th grade reading , 
writing and calculating level will be absolutely lost, 
(Dorothy Shields, Director of Education, AF£-eid) 
(Roueche, 1984) 

HU^L-TQ EVALUATE REMEDIAL PROGRAMS : 

Remedial/developmental programs are the fastest growing 
area of curriculum in colleges and universities. The question 
of how to conduct the evaluation of remedial programs .is a concern; 

At a recent conference on The Evaluation of Remedial Programs 
in Post Secondary Education held at Asilbmar> California/ in 
February, 1984, the California Community Colleges delineated some 
issues and implicatibhs of the evaluatibh bf remedial programs. 
Some of these issues and implications are summarized as follows: 

I. REVIEW OF ISSUES : 

. Respect for diversity of community colleges 
.Purposes of evaluation are questionable ^ 
.Manner in which evaluation results vy/ould be used 

• ?y?'P9?^_9f program evaluation will be to improve programs , . 
not to add or delete. __ _ ______ 

.bur responsibility will be coiiectiveiy to guide the external 
criteria. 

.Need balance between external and internal criteria- We will 
start with a bottom up approach to evaluation. 

• There is continuing interest in remediation , and no notion that 
we will give up this function. Expect no diminishing of this 
role or function.. 

.impatience of state level groups 

^Questions of who-vy/hat- hpvy/. Who will be invbived? 
.What is a program? 
.What data do we cbiiecr? 
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ll. EMERGING PROBLEMS : : 

.Legislature may invest more dollars in remediation^ not less, 

if we can prove benefits to the state. 
x.Pbssible new legislation on matriculation arid mandatory 

placement test. _ ' _ . 

.Negative impact of statewide evaluation system (resources , etc.) 

III. PRIO RITY CONCERNS : 

.To^se diversity* to develop effective' evaluation models. 
- .Purpose = program_ improvement . . . . 

-To define some common goals or criteria to establish 

comparability; _ ... 
.Involve all institutional family in participation in both' 

planni-ng and implementation - 
-Resources to do program evaluation- ^ 

Representatives at this conference proposed^ that these issues 

•will be 'reviewed by the Community Colleges of California in a series ^ 

* . _ . _ _ __ _ _ 

of activities that includes the following: 

i- Regional conferences, carefully planned. _ . . _^ 

'2- Appointment of -State Level Task Force to coordinate activities. 
3.. Survey of current practices — identify existing evaluation 

activities^in community colleges. . _ _ , , .__ 

4; Encourage on-site college visits to look at model activities. 
5; Use of outside consultant. _ _ _ 

6; Participation in . a Fall Conference (Model guidelines ^ill 

be established) . '_ . 

J. Implementation of Pilot evaluation* strat^ies to test feasibility. 

- While these issues are being addressed collectively, college 
administration need to develop an institutional agenda of activities. 
This agenda will include two types of activities: (1) the involvement 
of ^the total college in e^7aluatlon process; whether to utilize existing 
college cdminittees ^ LARC College Teams or to form a new committee will 
be institutional choices. However, the purpose should be to organize 
; for collective efforts not to fragment and compartmentalize; \2) the 
collection of information. . . 

The focus on the purpose 'of evaluation as a means for decision making 
has- become universally accepted Research notes that the primary purpose' 
of evaluating an education as training program is to provide information 
for decisions about programs- Such decisions should be useful for 
program improvement as well as for program termination - Useful infor--* . 
matioh cah.be extracted for • decisions regarding (1) program duplication^ 
(2) underproductive programs as well, as to ensure that program goals 
are being worked toward in the most efficient and "effective manner. 
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Based bri recent research and activities, the collection of 
information on remedial programs has included three types of 
information: (1) information related to programs and college 
goals, (2) . related to the content of programs, and (3) information 
on the effectiveness of courses and programs. 

When designing and evaluating such programs, administratbrs 
and faculty should consider the fbllbwihg: Studies indicate that 
successful learning skills programs are broadly described as having 
twb dimensibns:^ comprehensiveness and institutionalization and 
that the general expected payof f is increased GPA and r-iteni-^on. 

The check list below describes factors in the two dimensions 
that influence efficiency and effectiveness. These factbrs suggest 
the types of activities and information involved in program 
evaluation review. 

. . EVALUATION OF REraiAL PROGRAMS 

CHEGK LIST 

FACTORS INFLUENCING EFFICIENCY AND EFFECTIVENESS: 

COMPREHENSIVENESS : 

.Testing information that measures student progress along abilftj^ 
continuum; 

.Services currently 'available . 

-Testing programs (screening, placement, and diagnbstic) . 

.Academic areas (reading, writing, mathematics, and language skills) 

covered in developmental and basic skills programs . 

.Use of learning laboratories and aTjdio/visuai support j ^ 

.Availability of advisory services and staff development for facility. 
.The number of students in developmental versus basic skills programs. 



INSTITUTIONAL! ZATIQN : 

. Institutibnal commitment tb remediation. 
.Policies currently states. ; 

.Linkage of remediatibh to institutibnal mission to total cbllege 
prbgrams . 

.Statements oh cbmpetehcies . _ . . .. 

.Statements on butcbrries, long br shbrt duratibh emphasis. 

Instructibhal expertise in adult learning. * 
. Institutibnal missibh. 

.Faculty attitudes toward these students. 

•Relationship with other tutbrial and cbuhselirig services. 
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RETENTION: 

.Persistence rate . of students . 

. Irifdrmatidn> available oil retention. 

.Amburit of time spent in developmental studies dir basic skills 
courses befdre erirdlling iri college-level curriculum. 

.The attrition rate amdrq these students compared to the rest of 
the freshman class . 



PERFORMANCE : 

. Information on "student per f ormance in courses after taking develop- 
mental education. _^ 

. Comparisons oT these students 

.Of those students identified^ 

the first-year courses. 
.The number of identified students who also were referred to 

counseling center or other support services - 

CONCLUSION : 

The present and future most urgent need will continue to be 
the improvement of instruction. Hdwever> as we consider wlr at 
constitutes improved instruction, it is important to note that the 
improvement of instruction is not jUst an issue fdr remedial educa- 
tion. Remedial programs already are heavily evaluated. What we 
are to consider is learning imprdvement in all df the programs. We 
must look for the same elements of efficiency and effectiveness in 
the traditional academic disciplines and in the vdcatidnal instruc- 
tion as in the remedial programs. For improvement of programs means 
"We need td all dd well equally." 



with similar students who chose not 
the number who performed poorly in 
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